A Generalized Expression for the Gradient of the Fundamental Information Measures

This paper introduces a new functional approach to describe an expression for the gradient of mutual information with respect to the system parameters in general stochastic systems. In this approach the system input can be dependent on the system parameters in general cases. As an application of this model, it allows us to express the roles of interference and feedback and their interactions in a Multiple Access Channel (MAC) system. Using these results, we extend the mutual information and Minimum Mean Square Error (MMSE) relations to the additive Gaussian noise MAC with feedback. In this regard, it is shown that the gradient of mutual information can be decomposed into three distinct parts that are originated from MMSE, interference and feedback. Then, we interpret the destructive effect of interference and the constructive impact of feedback on this channel. Moreover, doing a comparative study we show the capability of the proposed functional approach to give more insights into the some of the available results in the literature. Index Terms Feedback, gradient of mutual information, multiple access channel, score function.